Wireless communications system and method

ABSTRACT

A wireless communication system includes a smart device configured for transcribing text-to-speech (STT) for display. The smart device interfaces with a radio communications device, for example, in an aircraft (AC). The system includes a filter for optimizing STT functions. Such functions are further optimized by restricting the databases of information, including geographic locations, aircraft identifications and carrier information, whereby the database search functions are optimized. Methods for wireless communications using smart devices and STT functionality are disclosed.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority in U.S. Provisional Patent ApplicationNo. 62/699,044 Filed Jul. 17, 2018, and also claims priority is U.S.Provisional Patent Application No. 62/715,380 Filed Aug. 7, 2018 both ofwhich are incorporated herein by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates generally to the field of wirelesscommunications, and in particular to an avionics communications systemutilizing speech-to-text (STT) functionality with an onboard smartdevice.

2. Description of the Related Art

One of the most common communication practices in aviation and aircraft(AC) navigation is voice communication via radio frequency (RF)transmissions. Examples include communications amongst aircrew, AirTraffic Control (ATC), Automatic Terminal Information Services (ATIS),etc. Aircrew are frequently tasked with managing AC piloting andnavigation while listening and responding verbally to ATC and ATIScommunications. Various systems and methods have previously beenproposed for managing and optimizing communications in aviationoperations. However, heretofore there is not been available a system andmethod with the advantages and features of the present invention.

BRIEF SUMMARY OF THE INVENTION

In the practice of the present invention, a wireless communicationsystem includes an RF communications radio configured for transmittingand receiving analog or digital voice communications. Without limitationon the generality of useful applications of the present invention, anonboard AC application is disclosed. An onboard smart device can beconfigured for personal use by an aircrew member or members. The smartdevice interfaces with the communications radio via a hardwired orwireless connection. The smart device includes a speech-to-text (STT)program configured for transcribing analog or digital communications anddisplaying them in textual format to an aircrew member. The systemfurther includes localizing functions for optimizing operation viaGNSS-defined AC locations, and a signal filter adapted to optimizespeech recognition functions in environments such as AC cockpits andcabins.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings constitute a part of this specification and includeexemplary embodiments of the present invention illustrating variousobjects and features thereof.

FIG. 1 shows the architecture of a communication system embodying anaspect or embodiment of the present invention.

FIG. 2 is a fragmentary, schematic diagram thereof, particularly showinga multi-channel system.

FIG. 3 is a fragmentary diagram of the system, particularly showing anaircrew headset, a smart (mobile) device and part of an AC cockpitpanel, hardwired together via a Y-splitter.

FIG. 4 is a fragmentary diagram of the system, particularly showing awireless interface between the smart device, an aircrew headset and theAC cockpit panel.

FIG. 5 is a fragmentary diagram of the system, particularly showing atraining system with a machine learning cluster.

FIG. 6 is a high level STT processing flowchart.

FIGS. 7-9 show alternative embodiment STT flowcharts.

FIG. 10 is a map showing an application of the localization function.

FIG. 11 is a Venn diagram showing call sign recognition probability.

FIG. 12 is a flowchart showing the high-level architecture of a trainingsystem.

FIG. 13 is a schematic diagram of an audio processing filter subsystem.

FIG. 14 is a block diagram of the system in conjunction with an ADS-Bcomponent.

FIG. 15 is a block diagram of a user interface of the system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

As required, detailed aspects of the present invention are disclosedherein, however, it is to be understood that the disclosed aspects aremerely exemplary of the invention, which may be embodied in variousforms. Therefore, specific structural and functional details disclosedherein are not to be interpreted as limiting, but merely as a basis forthe claims and as a representative basis for teaching one skilled in theart how to variously employ the present invention in virtually anyappropriately detailed structure.

Certain terminology will be used in the following description forconvenience in reference only and will not be limiting. For example, up,down, front, back, right and left refer to the invention as orientatedin the view being referred to. The words, “inwardly” and “outwardly”refer to directions toward and away from, respectively, the geometriccenter of the aspect being described and designated parts thereof.Forwardly and rearwardly are generally in reference to the direction oftravel, if appropriate. Said terminology will include the wordsspecifically mentioned, derivatives thereof and words of similarmeaning.

I. Introduction and Environment

Referring to the drawings more detail, the reference numeral 10generally designates a wireless communications system embodying anaspect of the present invention. Without limitation on the generality ofuseful applications of the system 10, an exemplary application is in anAC for facilitating aviation operations. The AC includes a cockpit panel12 mounting flight controls (e.g., yoke 14) and instrumentation 16,including a radio communications device 18 connected to a headset 20enabling an aircrew member (e.g., pilot) to effectively engage in RFcommunications.

As shown in FIG. 1, onboard components of the system 10 include theradio communications device 18 (shown in a radio/audio panel), the pilotheadset 20, an ADS-B device 22 (e.g., a Stratus device available fromAppareo Systems, LLC of Fargo, N. Dak. (U.S. Pat. No. 9,172,481 forAutomatic Multi-Generational Data Caching and Recovery, which isincorporated herein by reference)) and a microprocessor-based smartdevice 24.

Elements of the system remote from the AC (e.g. ground-based or onanother AC) include a server 26, a quality assurance operator 28, asoftware application source (e.g. Apple App store) 30. The server 26connects via the cloud (Internet) 32 to a Wi-Fi network 34 forconnection to the onboard microprocessor 24. A global navigationsatellite system (GNSS) source 33 provides positioning signals to thesystem 10. FIG. 1 shows communications paths for analog and digitalcommunications among the system 10 components.

FIG. 2 shows a multi-channel or multi-frequency embodiment of thepresent invention. A single-channel (frequency) embodiment is feasible,but RF communications systems commonly utilize multiple channels foraccommodating different types of communications (e.g., AC-AC, AC-ATC,emergency, weather, etc.). The radio/audio panel component 12/18accommodates the radio communications device 18 and other cockpit panel12 components. The STT box, as described above, can be coupled with themicroprocessor device 22. Audio input from the AC to the STT can also beprovided via an area microphone 36. FIG. 2 also shows the wired and/orwireless interfaces between the STT box 22 and the mobile/smart device24.

FIG. 3 shows the hardwired embodiment of the invention with the headset20, the smart/mobile device 24 and the radio communications device 18interconnected via a Y connector 36, which functions as a STT harness.Suitable plug-type, multi-conductor connectors can be utilized forefficiently assembling and disassembling the system 10 components. Forexample, aircrew often have their own headsets and smart devices, whichcan thus be transferred among multiple AC. FIG. 4 also shows thesecomponents, with a wireless audio input/output configuration with thesmart/mobile device 24.

FIG. 5 shows a training system for the system 10 including a machinelearning cluster 38 receiving inputs from an untrained model 40 andtraining data 42. The machine learning cluster 38 provides a trained 44model as output.

FIG. 6 shows a flowchart for the STT processing function of the system10. FIGS. 7-9 show flowcharts for alternative STT processing procedures.Such alternative procedures can be chosen for effectiveness inparticular applications. For example, AC applications may benefit fromcabin noise filtering. Moreover, STT can be optimized by accounting forregional dialects among speakers, multi-lingual STT software,localization and context-based speech recognition models.

FIG. 10 shows a database localization function based on GNSS-definedlocations of the AC. For example, on a cross-country flight,communications with ATC's, other AC, etc. are of greater interest inproximity to the AC's flight crew. Waypoints and fixtures are commonlyidentified with 4-5 characters for decoding. Thus, the current locationof the AC can define a circular geographic area of interest with apredetermined radius (e.g., 250 nm). Such filtering and localization canalso be accomplished by utilizing names of carriers, e.g., “FedEx” for adatabase subset corresponding to locations (airports, addresses, otherbusinesses, etc.) serviced by the Federal Express Corporation. Stillfurther, the database can utilize the tail number registrations assignedby the Federal Aviation Administration (FAA) for callsign localization.The system of the present invention utilizes these and other databasefunctions for maximizing the probabilities of accurate identifications.Such probabilities can be modeled and effectively utilized by softwarelocated onboard or remotely for access via the cloud 32. FIG. 11 shows ahierarchy with hypothetical probabilities based on databases including:all call signs; localized call signs; and air and ADS-B traffic callsigns. Other signals can be filtered and excluded. The operatingefficiency of the system 10 can thus be optimized by focusingconsideration of communications locally on a relatively small subset ofcommunications nationwide or globally.

FIG. 12 shows a flowchart for implementing the present invention usingspeech corpora, which can be selected among multiple options,normalizing data sets, data augmentation, deep neural networkbi-directional long short-term memory (LSTM) and Correctionist TemporalClassification (CTC), resulting in a trained language model output.

FIG. 13 shows a filter subsystem flowchart 52 configured for use withthe present invention. Signals progress from an audio frame 54 to awindow 56 and then to a fast Fourier transform (FFT) 58. From atruncated spectral slope and power ratio step 60 the method proceeds toa decision box 62 whereat the signal is analyzed for a rolling offcharacteristic. If “YES,” the method proceeds to a Zero Past Samplesstep 64 and then proceeds to a Zero Current Samples step 66. Through aqueue zeroed samples process the method proceeds to a Sample Queue at68, then to a Look Ahead Full decision box 70. If “YES,” speechrecognition results at 72. If the signal being filtered is not rollingoff, the method proceeds to a “Is Cabin” decision box: if no, the methodproceeds to the sample queue step 68; if yes, the method proceeds to adebounce step 76, satisfying the debounce and proceeding to the zeropast samples step 64.

FIG. 14 is a block diagram showing a Stratus software development kit(SDK) 74, a Knox (wireless communications system) library 84 includingan STT library (e.g., C++ programming language) 86, a base Stratusecosystem 88. Other device libraries 90 can provide additional data.FIG. 15 is a block diagram of a user interface of the system.

Having this described the invention, what is claimed as new and desiredto be secured by Letters Patent is:
 1. A speech-to-text (STT) avionicstranscription system for transcribing radio frequency (RF)communications transmitted from and received on board an aircraft (AC),which system includes: a smart device configured for receiving saidcommunications; said smart device including a display visible toaircrew; and a STT program installed and running on said smart device,said program configured for converting said communications to digitaltext for display on said smart device.
 2. The avionics system accordingto claim 1, which includes: an radio communications device onboard saidAC and configured for transmitting and receiving said RF communications;said radio communications device configured for selective connection tosaid smart device; and an audio subsystem onboard said AC, connected tosaid radio communications device and configured for output of saidcommunications to aircrew.
 3. The avionics system according to claim 2,which includes: said radio communications device and said smart devicebeing interconnected via one of a hardwired or wireless connection; anda Y-connector including a first connection to said radio communicationsdevice, a second connection to said AC audio subsystem and a thirdconnection to said smart device.
 4. The avionics system according toclaim 3, which includes: said radio communications device comprising amultiple-channel communications radio; and said multiple-channelcommunications radio is configured for monitoring: a primarycommunications channel for communicating with air traffic control (ATC)and other AC; and a secondary channel for weather and emergencycommunications.
 5. The avionics system according to claim 7, whichincludes an automatic dependent surveillance-broadcast (ADS-B) deviceonboard said AC and connected to said radio communications device. 6.The avionics system according to claim 3, which includes a machinelearning cluster configured for training said STT functions, saidmachine learning cluster configured for receiving untrained model andtraining data input and providing trained model output.
 7. The avionicssystem according to claim 3 wherein said aircrew can access saidcommunications audibly and/or visually.
 8. The avionics system accordingto claim 3 wherein said STT system includes: cabin filtering, voiceactivity detection (VAD), feature extraction, inference-neural network(acoustic model), decoder language model and natural languageprocessing.
 9. The avionics system according to claim 3 wherein said STTsystem includes: said inference-neural network (acoustic model) receivesacoustic model inference processing input; and said decoder (languagemodel) receives language model decoding input.
 10. The avionics systemaccording to claim 3, which includes: a Global Navigation SatelliteSystem (GNSS) onboard said AC and configured for locating said AC; andand a database localization feature utilizing said GNSS-defined AClocation.
 11. The speech-to-text avionics system according to claim 1,which includes: a database server located remotely from said aircraftand configured for interfacing via the cloud said smart device.
 12. Thespeech-to-text avionics system according to claim 1, which includes alocalization function configured for optimizing communications based onAC proximity to respective geographic locations along AC flight paths.13. The speech-to-text avionics system according to claim 16 whereinsaid localization function includes a call sign hierarchy functionassigning probability percentages among all call signs, localized callsigns, air traffic ADS-B traffic.
 14. The speech-to-text avionics systemaccording to claim 1 wherein said onboard radio is configured formulti-frequency transmitting and receiving.
 15. The speech-to-textavionics system according to claim 1 wherein said onboard radio includesa primary frequency and multiple secondary frequencies, said radioconfigured for receiving and transmitting on multiple said frequencies.16. The speech-to-text avionics system according to claim 1, whichincludes: a global navigation satellite system (GNSS) subsystemconfigured for locating an aircraft with said avionics system on board;and said avionics system configured for optimizing performance bylocalizing said aircraft with said GNSS-defined position data.
 17. Aspeech-to-text (STT) avionics transcription system for transcribingradio frequency (RF) communications transmitted from and received onboard an aircraft (AC), which system includes: a smart device configuredfor receiving said communications; said smart device including a displayvisible to aircrew; a STT program installed and running on said smartdevice, said program configured for converting said communications todigital text for display on said smart device; an radio communicationsdevice onboard said AC and configured for transmitting and receivingsaid RF communications; said radio communications device configured forselective connection to said smart device; an audio subsystem onboardsaid AC, connected to said radio communications device and configuredfor output of said communications to aircrew; said radio communicationsdevice and said smart device being interconnected via one of a hardwiredor wireless connection; a Y-connector including a first connection tosaid radio communications device, a second connection to said AC audiosubsystem and a third connection to said smart device; said radiocommunications device comprising a multiple-channel communicationsradio; and said multiple-channel communications radio is configured formonitoring: a primary communications channel for communicating with airtraffic control (ATC) and other AC; and a secondary channel for weatherand emergency communications; an automatic dependentsurveillance-broadcast (ADS-B) device onboard said AC and connected tosaid radio communications device; a machine learning cluster configuredfor training said STT functions, said machine learning clusterconfigured for receiving untrained model and training data input andproviding trained model output; said system configured for aircrewaccessing communications audibly and/or visually; said STT systemincluding: cabin filtering, voice activity detection (VAD), featureextraction, inference-neural network (acoustic model), decoder languagemodel and natural language processing; said STT system including:inference-neural network (acoustic model) receiving acoustic modelinference processing input; and a decoder (language model) receivinglanguage model decoding input; a Global Navigation Satellite System(GNSS) onboard said AC and configured for locating said AC; and and adatabase localization feature utilizing said GNSS-defined AC location.18. A wireless communications method including the steps of: providing asmart device configured for receiving said communications; providingsaid smart device with a display visible to aircrew; and installing onsaid smart device an STT program and running on said smart device, saidprogram configured for converting said communications to digital textfor display on said smart device.
 19. The method according to claim 19,which includes the additional steps of: providing a radio communicationsdevice selectively connected to said smart device; and optimizingperformance of said wireless communications method using one or more of:localization techniques, aircraft identification in an avionicsapplication, global navigation satellite system (GNSS) positioninginformation.
 20. The method according to claim 18, which includes theadditional filtering steps of: providing an audio frame for receivingaudio signals; defining a signal window; applying a fast Fouriertransform (FFT) to said audio signals; applying a truncated spectralslope and power ratio filter to set audio signals; if the signal beingfiltered is rolling off, proceeding to a zero pass samples step, thenproceeding to a zero current samples step, queuing the zeroed samples,looking ahead full and applying speech recognition; and if the signalbeing filtered is not rolling off, proceeding to a “Is Cabin” decisionbox: if no, proceeding to the sample queue step; if yes, proceeding to adebounce step, satisfying the debounce and proceeding to the zero pastsamples step.